CLDSRV-623: Bump arsenal for perf around error by BourgoisMickael · Pull Request #5761 · scality/cloudserver

BourgoisMickael · 2025-03-14T18:17:10Z

Check by commit

bert-e · 2025-03-14T18:17:13Z

Hello bourgoismickael,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

BourgoisMickael · 2025-03-17T13:17:43Z

lib/api/apiUtils/authorization/permissionChecks.js

+        As seen in https://scality.atlassian.net/browse/ARSN-477
+        the `isValid` does the same parsing as `parseCIDR` but
+        encapsulate in try catch to return a boolean.
+        The `isValid` in catch is useless and the double try
+        discards completely any error from ipaddr.parseCIDR.
+
+        Meaning in case of invalid ip there will be 2 ipaddr.js parsing,
+        reducing perf for an ignored error.
+
+        Fixing this breaks the tests.
+
+        I'm not even sure of the purpose of using ipaddr.js
+        and using our regex after as our regex seems to be
+        more restrictive.
+
+        ipaddr.js accepts valid ip like:
+            - 0010.0xa5.1.1
+            - ::ffff:222.1.41.900


An if instead of a catch for isValid should be used, but do we really need to use ipaddr.js ?

I think we can try to drop it but we need to be careful that it doesn't break the IP conditions in IAM/Bucket policies. These policies can test ranges and mix of ipv4 & ipv6 addresses.

Note: this was implemented 1 year ago here: https://github.com/scality/cloudserver/pull/5501/files#diff-773344fcfc0f64145ff95eb6eccd556588c19449dc2c263a2532d4030bf00dffR490-R515

Since then the ipaddr.js parsing error has always been ignored due to the double try.

So removing it won't change the behavior but improve perf in case of invalid IP.

Now I don't know if we should handle special variations of ipv4 or ipv6 like it's done by ipaddr.js: https://www.npmjs.com/package/ipaddr.js/v/2.2.0?activeTab=code

And maybe our ipv6 regex is missing the cidr suffix: https://github.com/scality/cloudserver/blob/development/9.0/constants.js#L236-L237 ?

Now I don't know if we should handle special variations of ipv4 or ipv6 like it's done by ipaddr.js

We control the ip we get in the request, from the code we have in arsenal, so as long as we guarantee the right IP format is returned, we are fine. In AWS, the condition either supports ipv4 or ipv6: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceip.
So I would say it's fine not to use ipaddr, as long as the regex is enough to ensure the lib will accept it in the actual logic (in Arsenal).

And maybe our ipv6 regex is missing the cidr suffix

Indeed.

I'll drop the commit about ipCheck and have another ticket to compare with aws if the current regex is good and if we need to handle special ip

Dropped and ticket is https://scality.atlassian.net/browse/CLDSRV-627

package.json

lib/api/apiUtils/authorization/permissionChecks.js

ghost · 2025-03-17T16:46:08Z

lib/api/apiUtils/authorization/permissionChecks.js

+        As seen in https://scality.atlassian.net/browse/ARSN-477
+        the `isValid` does the same parsing as `parseCIDR` but
+        encapsulate in try catch to return a boolean.
+        The `isValid` in catch is useless and the double try
+        discards completely any error from ipaddr.parseCIDR.
+
+        Meaning in case of invalid ip there will be 2 ipaddr.js parsing,
+        reducing perf for an ignored error.
+
+        Fixing this breaks the tests.
+
+        I'm not even sure of the purpose of using ipaddr.js
+        and using our regex after as our regex seems to be
+        more restrictive.
+
+        ipaddr.js accepts valid ip like:
+            - 0010.0xa5.1.1
+            - ::ffff:222.1.41.900


I think we can try to drop it but we need to be careful that it doesn't break the IP conditions in IAM/Bucket policies. These policies can test ranges and mix of ipv4 & ipv6 addresses.

lib/api/apiUtils/authorization/permissionChecks.js

ghost

LGTM
There is a merge commit that should be properly removed by rebasing the PR (it's hard to review with it)

BourgoisMickael · 2025-03-20T14:50:58Z

LGTM There is a merge commit that should be properly removed by rebasing the PR (it's hard to review with it)

@williamlardier I only see my changes on this merge commit, I don't see the other changes it brings in the review

ghost · 2025-03-20T14:53:37Z

Maybe a github display issue, all good if we see only your changes 🙂

bert-e · 2025-03-20T22:47:56Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

BourgoisMickael · 2025-03-20T22:48:07Z

/create_integration_branches

BourgoisMickael · 2025-03-20T23:58:09Z

/create_pull_requests

bert-e · 2025-03-20T23:58:23Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge improvement/CLDSRV-623-arsn-bump-perf into
development/7.70
w/8.8/improvement/CLDSRV-623-arsn-bump-perf will be merged into development/8.8 (pull request INTEGRATION [PR#5761 > development/8.8] CLDSRV-623: Bump arsenal for perf around error #5771)
w/9.0/improvement/CLDSRV-623-arsn-bump-perf will be merged into development/9.0 (pull request INTEGRATION [PR#5761 > development/9.0] CLDSRV-623: Bump arsenal for perf around error #5772)

The following branches will NOT be impacted:

development/7.10
development/7.4

Follow integration pull requests if you would like to be notified of
build statuses by email.

The following options are set: create_pull_requests, create_integration_branches

bert-e · 2025-03-20T23:58:24Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_pull_requests, create_integration_branches

BourgoisMickael · 2025-03-21T15:27:01Z

/approve

bert-e · 2025-03-21T15:27:20Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/7.70
✔️ development/8.8
✔️ development/9.0

The following branches have NOT changed:

development/7.10
development/7.4

Please check the status of the associated issue CLDSRV-623.

Goodbye bourgoismickael.

The following options are set: approve, create_pull_requests, create_integration_branches

BourgoisMickael force-pushed the improvement/CLDSRV-623-arsn-bump-perf branch 5 times, most recently from 32ac11a to 3c1ba0c Compare March 17, 2025 12:54

BourgoisMickael commented Mar 17, 2025

View reviewed changes

ghost reviewed Mar 17, 2025

View reviewed changes

BourgoisMickael requested review from a user, anurag4DSB, dvasilas, fredmnl, jonathan-gramain and tmacro March 18, 2025 18:24

BourgoisMickael force-pushed the improvement/CLDSRV-623-arsn-bump-perf branch from 70ef4e8 to 3c09b1a Compare March 20, 2025 11:21

ghost approved these changes Mar 20, 2025

View reviewed changes

dvasilas approved these changes Mar 20, 2025

View reviewed changes

BourgoisMickael added 3 commits March 20, 2025 23:34

CLDSRV-623: Bump arsenal for perf

fb21926

CLDSRV-623: Avoid duplicate error instanciation

3c128f8

CLDSRV-623: Use errorInstances in tests

94d0ffe

BourgoisMickael force-pushed the improvement/CLDSRV-623-arsn-bump-perf branch from e88ce9f to 94d0ffe Compare March 20, 2025 22:34

scality deleted a comment from bert-e Mar 20, 2025

CLDSRV-623: Bump version

d4933b9

BourgoisMickael changed the title ~~CLDSRV-623: Bump arsenal for perf around error & ipcheck~~ CLDSRV-623: Bump arsenal for perf around error Mar 20, 2025

BourgoisMickael mentioned this pull request Mar 20, 2025

CLDSRV-623: Bump arsenal for perf around error |HF 9.4.3] #5770

Merged

bert-e mentioned this pull request Mar 20, 2025

INTEGRATION [PR#5761 > development/8.8] CLDSRV-623: Bump arsenal for perf around error #5771

Closed

bert-e mentioned this pull request Mar 20, 2025

INTEGRATION [PR#5761 > development/9.0] CLDSRV-623: Bump arsenal for perf around error #5772

Merged

scality deleted a comment from bert-e Mar 20, 2025

bert-e merged commit d4933b9 into development/7.70 Mar 21, 2025
11 checks passed

bert-e deleted the improvement/CLDSRV-623-arsn-bump-perf branch March 21, 2025 15:27

Conversation

BourgoisMickael commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bert-e commented Mar 14, 2025

Hello bourgoismickael,

Uh oh!

BourgoisMickael Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

ghost Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

BourgoisMickael Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

ghost Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

BourgoisMickael Mar 20, 2025

Choose a reason for hiding this comment

Uh oh!

BourgoisMickael Mar 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

BourgoisMickael commented Mar 20, 2025

Uh oh!

ghost commented Mar 20, 2025

Uh oh!

bert-e commented Mar 20, 2025

Request integration branches

Uh oh!

BourgoisMickael commented Mar 20, 2025

Uh oh!

BourgoisMickael commented Mar 20, 2025

Uh oh!

bert-e commented Mar 20, 2025

Integration data created

Uh oh!

bert-e commented Mar 20, 2025

Waiting for approval

Uh oh!

BourgoisMickael commented Mar 21, 2025

Uh oh!

bert-e commented Mar 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

BourgoisMickael commented Mar 14, 2025 •

edited

Loading